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Advanced biomedical engineering technologies are continuously changing 
the medical practices to improve medical care for patients. Needle insertion 
navigation during intravenous catheterization process via Near infrared 
(NIR) and camera-projector is one solution. However, the central point of 
the problem is the image captured by camera misaligns with the image 
projected back on the object of interest. This causes the projected image not 
to be overlaid perfectly in the real-world. In this paper, a camera-projector 
calibration method is presented. Polynomial algorithm was used to remove 
the barrel distortion in captured images. Scaling and translation 
transformations are used to correct the geometric distortions introduced 
in the image acquisition process. Discrepancies in the captured and projected 
images are assessed. The accuracy of the image and the projected image 
is 90.643%. This indicates the feasibility of the captured approach to 
ehminate discrepancies in the projection and navigation images. 
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1. INTRODUCTION 

Intravenous injection is difficult and painful especially for newborns, children, and obese people 
because appropriate veins maybe deep and not visible. The apphcation of near-infrared hght (NIR) has been 
a good way to help locate vein, since the hght can penetrate up to several centimeters into tissues and be 
absorbed by the vein, which enables discrimination between blood vessels and surrounding tissues [1-4]. 
Based on the theory, several devices have been developed recently to aid physicians, phlebotomists and 
surgeons in locating veins to avoid unnecessary sticks. 

Some researchers and companies have developed devices which can show an enhanced image on 
the screen to help locate vein [5-10]. However, it is not convenient for the phlebotomist to watch the screen 
and do the vein puncture simultaneously. Therefore, the concept of camera-projector system has been 
introduced to overcome the problem. However, the central point of the image being captured by camera 
is different than that being projected back. It causes the projected image not to overlay perfectly with the real 
object. Therefore, it is necessary to calibrate camera and projector. 

The first implementation of camera-projector system apphed high precision micrometer screw 
adjusted via mechanical stages to set the optimum position of camera and projector. Then, a simple 
transformation functions (translation, scale and rotation) was used to ahgn the images on the object [11-12]. 
Later, was a different method proposed by [13] had improved method developed by [11]. The difference 
is the placement of the camera, projector and NIR LEDs source in the same axis to remove the hot mirror. 
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However, the placement of each component in different locations requires a large space. It also limits 
the projection area to the region surrounded by the black rectangular plate. In addition, similar back 
projection method was proposed by [14] using dual camera system. This method was able to obtain the vein 
depth, but the projection result was similar to single camera system and need huge computation time for 
the imaging process. 

The work by [15] observed that the previously developed projection methods only perform 
back-projection at a certain distance without observing the results at difference distances. 
Therefore, [15] proposed a projection method using the intersection between camera view and projection 
view technique attached with the distance sensor as a parameter to adjust the projection. 
However, the implementation of the projection method is offline. To be different from others, [16] proposed 
a combination of NIR imaging technique and augmented reality (AR) technology in head mount device 
(HMD). This HMD system created a virtual image and overlay onto real-world object to form a composite 
view in the HMD see-through lenses. However, the adjustment camera mount (IR CCD camera) and epipolar 
geometry are complex and can lead to complex view. 

By now, AccuVein AV300 [17] is the most advanced vein display device. However, the structure 
and the program of the device is intricacy and the cost is very expensive. Therefore, this work proposes 
to solve the problem of the vein display focusing on optimal camera-projector setting. This paper 
is structured into five sections. The overview of the camera-projector calibration techniques and the problems 
encountered are discussed in this section. Section 2 and 3 presents the system structure and the proposed 
camera-projection calibration. Then the results obtained are presented and discussed in section 4. 
Lastly, section 5 concludes the findings of the research work. 


2. SYSTEM STRUCTURE 

This system consists of pico-projector, night vision camera, three sets of 850nm high power NIR 
LED array, diffuser, based as a target area, raspberry pi 3 B-i- and aluminum rod as the main structure. 
Three high power 3W infrared LEDs are focused on the target area and mounted to the left, right and front of 
the camera. The camera and LEDs are connected to Raspberry Pi with flex cable. The Raspberry Pi and NIR 
LEDs are derived by a 5V 2.5A power supply. The illumination is almost constant during the experiment 
with the help of a diffuser located in front of each NIR LEDs. The camera is connected to Raspberry pi by 
way of a short ribbon cable. At another end, the camera is connected to BCM2837 processor on the PI via 
the CSI bus. 

The pico-projector is placed at the top of the object, at a distance of 45cm from the base. The light 
sources are placed around the camera. The reflected light by NIR LEDs is captured by the cameras 
on the same side. The distance between the target area and the camera is set to 32cm. These distances are 
chosen as the best positions based on a number of trial-and-error experiments to get better overlapping before 
proceeding on with the lens distortions, scaling and translation. A printed 7x7 chessboard patterns as a test 
image is placed on the base. The night version camera is connected to Raspberry Pi board using a CSI cable 
and pico-projector is connected to the Raspberry Pi using HDMI. After capturing the raw image from 
the NIR camera module, the images have been processed by a portable Raspberry Pi 3 B-i-microcomputer 
using Open-CV image processing library codedin Python programming language. 


3. CAMERA-PROJECTOR CALIBRATION 

The overall proposed camera-projector calibration procedures are illustrated in Eigure 1. It involves 
three stages; lens distortion correction, scaling transformation and translation transformation. Lens distortion 
correction has been employed to remove barrel distortion in captured image. Scaling and translation 
transformations are used to correct geometric distortions introduced in the image acquisition process. 

3.1. Lens distortion correction 

Captured image normally suffered from lens distortion. Based on the literature review by [18-19], 
the dominant lens distortion is usually barrel distortion. This is due to the imperfection of the lens 
and the misalignment of the optical elements. Many lens distortion corrections have been proposed from 
a simple to a complex algorithms. Polynomial algorithm has been used in this work due to its simplicity 
and accuracy [20-22]. The algorithm starts with (xd, yd) as the measureable coordinates of the distorted image 
point, (xu, ju) the coordinates of the undistorted image point. The polynomial algorithm commonly used to 
describe radial distortion can be written as [18]: 

+■■ ■) ( 1 ) 
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where Vd and Vu are the distances of the distorted point (xd, yd) and the undistorted point (xu,yu) to the distorted 
center P respectively, and k is the parameter of the radial distortion. 



Lens distortion coirection 


yr 


Scaling tiansfoniiation 




Translation tiansfoniiation 



Figure 1. Stages of the proposed camera-projector calibration work 


For practical purpose as being explained by [23], only the first term of the infinite series has been used, 
which is: 


r„ = (1 + (2) 

In this work, the contribution of k value towards the appearance of the corrected lens distortion image is 
investigated. The method of determining the optimum value for k is based on trial-and-error. 


3.2. Scaling transformation 

The problem in this stage is the central point of the image by the camera being different from 
the one projected back by the projector. Hence, the size of both images normally becomes different. 
Therefore, image scaling has been applied. Scaling is a linear transformation that is able to enlarge or 
shrinks’ objects by a scale factor that is the same in all directions. An x-value defined the amount of scaling 
in the x direction, and a y-value defines the amount of scaling in the y direction. For each pixel (x,y) 
of the destination, the source value at the fractional subpixel position is constructed by means of an 
interpolation object and written to the destination [24]. The coordinates are given by: 

x' = - (3) 


where is the x scale factor and is the y scale factor. When applying scale factors to a source image with 
width ofw^ and height of/I 5 , the resulting image is defined to have the following dimensions [24]: 


qwa (5) 

ha=hs^ Vs ( 6 ) 

Scale factor values greater than 1.0 magnify the image, while value less than 1.0 minify the image. 
In order to understand the relationship between scale factors and the resultant image, a simple experiment 
is carried out. The method of obtaining the optimum value for scale factors is based on trial-and-error. 
Segmented image into three cluster has been used to evaluate the performance of resulted image. The cluster 
image can be categorized into three regions; black, grey and white pixels. The black pix el represents the real 
chessboard patterns region, grey pixel represents the projected chessboard region and white pixels represent 
the background region of the image. From the clustered image, the proposed technique has been 
quantitatively evaluated by calculating the deviations between the total number of black and grey pixels. 
The deviations are then calculated as follows: 


Deviation = 


Z black pixels -Z grey pixels 
Z black pixels 


(7) 
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3.3. Translation transformation 

The projected chessboard patterns are not only distorted in terms of size but also the position 
of the image. Therefore, image translation has been applied after scaling transformation. Image translation 
is the spatial shifting of an image up, down, left or right. The relationship between the source and destination 
image coordinates are given by the following equation [25]: 



(8) 

Vd = ys + ty 

(9) 


where and are the integer pixel coordinates of the destination image, and ty are the translation 
values, X 5 and denote the source image point from which the pixel estimate is computed. The projected 
image also has been converted into three clustered for more precise qualitative and quantitative analysis. 
The values of and y^ are determined based on trial-and-error depending upon image. 

While, for quantitative analysis, is based on overlapping percentages between total number of black and 
non-overlap grey pixels. The overlapping percentage is calculated using: 

, . black pixelsnon-overlap grey pixels . 

overlapping percentages = - z black pixels -^ 

In image acquisition process, the object region of interest is the middle of the chessboard pattern as 
shown in Eigure 2. Therefore, in this work, the best resulted obtained from previous analysis (overlapping 
percentages) has been divided into three parts; inner, middle and outer parts. The red square represents 
the inner part, yellow square represents the middle parts and blue square represents the outer parts. 
The overlapping percentages for these three part are obtained using (10). 



Eigure 2. Chessboard patterns with three regions; inner, middle and outer 


4. RESULTS AND DISCUSSION 

Eigure 3(a) shows the checkerboard pattern captured using NIR camera and Eigure 3(b) 
is the resultant image projected back onto the real printed chessboard using pico-projector. In Eigure 3(b), 
it can be seen that the projected chessboard pattern is actually the one which looks like a shadow and 
the black is the real chessboard pattern. This is due to the central point of the image being captured by 
the camera is different than that being projected back from the projector. As shown in Eigure 3 it can be 
observed that the lines of the chessboard patterns are bulging and its image appears as more magnified at 
the center than that at the edges due to the barrel distortion. 

Eigure 4 shows the results of chessboard patterns after lens distortion correction with different 
values of distortion coefficient, k. Erom the results, it is observed that, before k=-0.12 the lines of 
the chessboard patterns appear straight at the very center of the frame and only start bending away from 
the center (refer to Eigure 4 (b) and (c)). However, after I = -0.72, aU corrected lines do not go through 
the center of the image are bowed inwards, towards the center of the image like pincushion 
(refer to Eigure 4 (e) and (f)). Therefore, it can concluded that, the best lens distortion correction image 
is obviously atk=-0.72. 
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Figure 3. Image, (a) Of the chessboard pattern taken usingNIR source and, (b) Projected back onto real 

printed chessboard using pico-projector 



(e) (f) 

Figure 4. Results of applying lens distortion correction on, (a) Distorted input image with distortion 
coefficient of, (b) k=-0.06, (c) k=-0.09, (d) k=-0.12, (e) k=-0.15 and(f) k=-0.18 
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Table 1 shows the result of scaling the projected chessboard pattern with different values of scaling 
ratio. The projected image is the one which is in shadow color and the black color is the real chessboard 
patterns. Cluster image is the images after segmentation process to reduce the difference between real and 
projected chessboard patterns. As can be seen, at scaling ratio of 1.00 and 1.15, the size of the grey patterns 
seems significantly small compared to the black patterns. This means that the scaling ratios of 1.00 and 1.15 
are not enough to scale-up the size of the projected patterns. Scaling ratio of 1.45 gives bigger size to the 
projected patterns. Only when scaling ratio is 1.30, the outputs of the projected and real patterns are almost 
similar in size. The above results is also supported by deviations between the number of pixels of real 
patterns and projected patterns as shown in Table 2. The smallest deviation of pixels for both regions is at 
scaling ratio value of 1.3. Table 3 shows the results of investigation towards determining the optimum values 
of X andy-axes. 


Table 1. Projected images of chessboard onto real image patterns with different values of scaling ratio 
Scaling 
Ratio 


Projected Image 


Cluster Image 
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Table 2. Deviation in pixels of scaling ratio between real and projected pixels 


Scaling 

Ratio 

Area calculated in pixels 

- Deviation in pixels 

Real ROI (black pixels) 

Proiected ROI (grey pixels) 

1.00 

6724 

5476 

0.186 

1.15 

6724 

5625 

0.163 

1.30 

6241 

6561 

0.051 

1.45 

6241 

7741 

0.241 


Table 3. Results for NIR image translation for different y-coordinate values when x-axis value is at 3 
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In order to analyses the results easier, the NIR image has been segmented to cluster image. 
The cluster image can be categorized into three regions; black, grey and white pixel. The black pixel 
represents the real chessboard patterns region, grey pixel represents the projected back region and white 
pixels represented the background region of the image. As can be seen in overall results, when the values of 
y-axis are at -30 and -40, the projected regions (grey pixels) are located towards the bottom from the real 
chessboard regions (black pixels). While y-axis is equal to -70, the projected regions are located towards to 
the upper of the real chessboard regions. Hence, from the observation, the projected region and real 
chessboard pattern region almost perfectly overlap to each other when x-axis equals to 3 and y-axis equals 
to -50 compared to other results. This results are also supported by the overlapping percentages as shown in 
Table 4. The x-axis=3 and y-axis=-50 produced the highest overlapping percentages of 90.643%. This shows 
that the values of x and y axes at that point is the most optimum for translation transformation. 
Next, the optimum translation values of x and y axes can be analyzed more details by dividing the cluster 
image into 3 regions; inner, middle and outer. This is due to the future process, where the ROI will be 
focused on the center of the cluster image, which is the inner and middle regions. 


Table 4. Overlapping percentages for each values of x and y-axes translation 


X-axis 

Y-axis 

Total number of real pixels 
(black) 

Total number ofnon-overl^ 
proiected pixels (grey) 

Overlapping percentages (%) 


-30 

129346 

54073 

58.195 


-40 

129407 

41024 

68.298 

-3 

-50 

129586 

25910 

80.006 


-60 

129810 

26428 

89.641 


-70 

129510 

35788 

72.367 


-30 

129231 

51676 

60.013 


-40 

129431 

37973 

70.662 

0 

-50 

129396 

20670 

84.026 


-60 

129612 

19376 

85.051 


-70 

129483 

35618 

72.492 


-30 

128986 

53666 

58.394 


-40 

129193 

37797 

70.744 

3 

-50 

129090 

12079 

90.643 


-60 

129498 

17442 

86.531 


-70 

129383 

35292 

72.723 


-30 

128820 

52437 

59.294 


-40 

128921 

37818 

70.666 

6 

-50 

129209 

24505 

81.035 


-60 

129375 

22138 

82.889 


-70 

129421 

37487 

71.035 


-30 

128701 

55029 

57.243 


-40 

129071 

41000 

68.235 

9 

-50 

129171 

27839 

78.448 


-60 

129140 

16288 

79.644 


-70 

129294 

38697 

70.071 


Table 5 shows the overlapping percentages for inner, middle and outer regions of the chessboard 
patterns at optimum x and y-axes translation. As can be seen, the inner region produced the highest 
overlapping percentage, 92.915% followed by middle region with 90.925%. Although the outer region is not 
the focus of this work, its overlapping percentage also approaches to 90%. Eigure 5 shows the results of test 
objects projected onto the chessboard patterns. As can be seen, the test objects have been successfully 
projected on and aligned. 


Table 5. Overlapping percentages for inner, middle and outer regions of chessboard patterns at translation 
_ x=3 andy=-50 _ 


Region 

Areas calculated in pixels 

Black pixels Non-overlap grey Pixels 

0 V erlapp ing p ercentage 

Inner 

24968 

1769 

92.915 

Middle 

41994 

3811 

90.925 

Outer 

59934 

7138 

88.090 
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Figure 5. A real test objects to verified the successful camera-projector calibration and alignment 


5. CONCLUSION 

This paper presented method to calibrate a camera-projector system to be used in a vein 
visualization system without any high precision mechanical parts. First, the camera-projector system was 
applied with lens distortion correction to remove the barrel distortion in captured image by using the simplest 
technique of polynomial algorithm. Then, scaling and translation transformation were used to correct 
geometric distortions, so that the projected image is overlaid onto the exact location of object 
in the real-world. This method is formed feasible and able to solve the camera-projector calibration problem. 
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